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O.S.S.E. 



■ Observing System 

■ Simulation 

■ Experiment 


Model-based OSSE 


A framework for numerical experimentation in which 
observables are simulated from fields generated by an 
earth system model, including a parameterized description 
of the observational error characteristics. 

Simulations are performed in support of an experimental goal. 
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The Validation Imperative 



■ As with any simulation, OSSE results 
apply to new instruments only to the 
degree they have been validated with 
existing legacy instruments. 

■ OSSE credibility is first determined by 
carefully comparing a variety of statistics 
that can be computed in both the real and 
OSSE simulated contexts. 


OSSEs need to be validated as a System . 




Nature Run 



IESA: Integrated 
Earth System Modeling 




Balancing act: Complexity x Resolution x Spatial/Temporal Coverage 





Nature Run Realism 



Visible Clouds 
July 27, 2009 18z 
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Atmospheric Composition NR: N ^ SA 

Simulated or Observed Meteorology? ^ ' 



GEOS-5 10km Global Mesoscale Simulation: SST, aerosol emissions 
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Simulators and 
Observation Error 

Modeling 



OSSE: 

Types of Simulators 



■ Level 1 simulators 

Detailed radiative transfer calculation in the presence of 
clouds, aerosols, ice, etc. 

Instrument characteristics 

Observables: polarized radiances, backscatter 

■ Level 2 simulators 

Retrieved quantities at observation location 
Averaging kernels, error characteristics 

■ Level 3 simulators 

■ Hourly to seasonal mean statistics sampled at the 
instrument footprint 
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Observation Error 


ivr^SA 



at Instrument Footprint 


♦ ' 



Measurement equation: 


y = f(z *) + e 


where 

y measurement 

z l true state at instrument footprint 
/ observation operator 

e observation error w.r.t. z t (detector noise) 




Error of Representativeness N * s ' 4 


Let x f be the true state in model grid point space: 

z l = X{x x ) + € 

where 

X remapping operator 
e/ representativeness error 

In OSSE studies the Nature Run is assumed to provide the 
ground truth, but it does so in grid-point space: x l 

V — + eO + e 

= /i(x^) + e + F z e f + .. 

It is critical to account for error of representativeness errors 
when simulating observables with a footprint much smaller than 
the Nature Run resolution. 



Sub-Grid Variability 








Clouds & 

Sub-grid Variability 



PDF 



S = (q v + q L + qO / q s (T) 


PDF-based cloud 
parameterizations provide 
very useful information 
about sub-grid variability 
Given a PDF of total water 
one can generate sub- 
columns consistent with 
that PDF 

Observation simulators 
can account for 
representativeness error 
by operating on these sub- 
columns 




Hygroscopic Aerosols 


GOCART prognosticate aerosol dry 
mass mixing ratio q c \ry , with hu- 
midification effects being included 
diagnostically prior to computing 
optical depth 

r = P(RH; p) ■ q dvy • p a Sz 

The normalized mass extinction 
efficiency 



/3{RH) 

m 


humidification factor 



/ 3 saturates at 99% 


r\j 
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PDF-based Humidification 



Recall that aerosol satellite retrievals are only available 
under clear sky conditions: 

^clear = /^clear (-R-H") * ^dry * Pa$Z 


PDF-based cloud schemes as in GEOS-5 can be used 
to estimate the mean humidification effect on a GCM 
gridbox 


<P> = 


pOO 

/ P (S)/3(S)dS 

Jo 

pi pOO 

= J p{S)0(S)dS + J p(S)0(S)dS 

— (1 f ) " >clear t . f ' /? ^cloudy 


where the cloud fraction f is given by 



A PDF of water vapor 
+ condensate is 
provided in each 
gridbox 


/ oo 

p(S)ds 



Radiance Error Modeling 



Radiance errors (detector noise) e are defined as 

y = /(**,&*) +e 

where 

y radiance measurement 

/ forward function (radiative transfer function) 

z l true state (z l — Hw l ) 

b f true forward model parameters 

(e.g., spectral line data, calibration parameters) 
e detector noise + error of representativeness 

The real physics of the radiative transfer is often too complex 
or its details unknown. In practice, a forward model (F) is used 

f(z\ b\ b 0 = F(z\ 6 4 ) + 5f{z\ b\ b * ) 





Retrieval Error Modeling 



> Retrievals are produced as result of a nonlinear estimation, 

z = D(y, b, z?) = D{F(z\ b*) + Sf + e, b, z?) 

where z p is a background state (prior) used in the retrieval. 

> In order to obtain a basic understanding we linearize this equation around a 
known state, say the prior z p 

9-z p = [D(F(z p ,b),b,z p ) -z p ] 

+DyF z (z t - z p ) 

^-DyF^b 1 - b) 

where T(z p ,b ) — D(F(z p ,b),b, z p ) is the so-called 
Transfer Function. 

> The term in brackets is the transfer function bias and should be small for any 
reasonable retrieval, 

T{z p , b) = D(F(z p , 6), 6) w z p 

i.e., the retrieval should return the first guess when noiseless data consistent 
with it is input. 





Retrieva l Error Analysis 



Ignoring the transfer function bias for the prior states, the re- 
trieval error reads: 

e r = z-Hw 1 

= (/ - A)e p + D y F b e b + D y Sf + D y e 
where e p = z p — z l = H(w p — w t ), etc., and 

A averaging kernel (— D y F z ) 

(7 - A)e p smoothing error (prior error) 

D y F b e b forward model parameter error 

D y 8f forward model error 

D y e instrument + representativeness error 




Retrieval Error Mechanisms 



> Averaging Kernel. The retrieved state is a smoothed version of the true state 
with smoothing functions given by the rows of A 

z = Az t + (I — A)z p + ... 

Those details of the true state which are smoothed out by the retrieval must 
be provided by the first guess z p . Notice that A is state dependent. 

> Model parameter errors are usually associated with biases. If retrieval is too 
sensitive to a parameter then it should be retrieved as well (state augmenta- 
tion). 

t> Model error. Hard to evaluate. If correct physics is known then it can be 
modeled (e.g., approximations introduced for computational efficiency.) 

> Instrument error. Usually the easiest to evaluate. It can have a strong de- 
pendence on the state through D y , although the detector noise e is usually 
assumed stationary and state independent in clear sky. Cloud clearing intro- 
duces additional complications. However, error of representativeness is often 
state dependent. 




Simulating Retrievals % 


FROM RADIANCES 

■ Synthetic retrievals 

Simulate radiances by 
radiative transfer 

■ Model radiance errors 

Apply retrieval code 


BY MODEL SAMPLING 

■ Sample and perturb 

Interpolate geophysical 
to obs location 

■ Model retrieval errors 

Done. 


While interpolating a model simulated geophysical quantity 
to observation location is much more straightforward than 
performing a full RT calculation, modeling retrieval errors is 
far more complex than modeling radiance errors. 
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OSSE Experiments 



The "E" in OSSE 




Wx 

r~ aq 
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OSSE: Types of 
Experiments 



■ Forecasting/data assimilation impact 

■ Goal: assess impact of new instrument 

■ Metric: analysis or forecast impact 

■ Nature Run: a high-resolution model 
simulation by another model 

■ Calibration: observational errors for 
existing instruments are tuned to 
produce same impact as real Data 
Assimilation System (DAS) 
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OSSE: Types of 
Experiments 



■ Retrieval algorithm evaluation 

Goal: given radiances/backscatter at a variety of 
atmospheric conditions (clouds, rain, ice, etc.) assess 
the quality of 

■ Retrievals themselves 

■ Retrieval perceived errors/averaging kernels 

Example of metrics: direct comparison of retrievals 
and Nature Run, full set of statistics 

Example of Nature Run: global aerosol transport 
model driven by “observed” meteorology 

Calibration: when possible tune observational 
errors based on similar current instruments 
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OSSE: Types of 
Experiments 



■ Sampling Studies 

■ Goal: assess how a given orbit, scanning 
characteristic would impact the ability of the 
measurements to reproduce key climate 
parameters 

Example of metrics: hourly to seasonal time 
means of Level 2 parameters at a variety of 
spatial resolutions 

Example of Nature: global aerosol transport 
model driven by “observed” meteorology 

Calibration: noisy and perfect observations 
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OSSEs at NASA's 
Global Modeling and 
Assimilation Office 




7-km GEOS-5 Nature Run 


Global Modeling and Assimilation Office 


Nature Run Details 
2-years : June 2005 -June 2007 
7-km Global Resolution [72 vertical levels : O.Olmb top] 
5-minute physics time step [5 seconds for dynamics] 
Non-Hydrostatic Dynamics [Finite-Volume Cubed-Sphere] 
Limited deep convection [RAS with stochastic Tokioka limiter] 
Resolve mesoscale weather [storm-scale and cloud clusters] 
High-resolution constituent transport [GOCART] 

Executed on "Discover" at NCCS [7200 Cores : 11-days/day] 


Surface Boundary Conditions & Emissions 

SST and Sea-Ice [1/4-degree Reynolds/OSTIA] 

C0/C0 2 Fossil fuel emissions [10-km EDGAR inventory] 

Land C0 2 fluxes [CASA-GFED at 10-km with MODIS EVI] 
Biomass burning [daily QFED emissions] 

Volcanic S0 2 [AEROCOM emissions and injection heights] 
GOCART [mixing, chemistry, and deposition] 

- Aerosol species [sulfates, dust, sea-salt and black carbon] 

- aerosols are radiatively coupled with the dynamics 





Surface C0 2 Concentration: 

The surface concentration of C0 2 highlights the fidelity of local emission centers and 
constituents being dispersed within the regional meso-scale flow. 




■ 





v 
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Column C0 2 Concentration: 

Surface emissions are lifted throughout the column and transported beyond the regional 
scales within the planetary flow. 



7-km GEOS-5 Nature Run 

Global Modeling and Assimilation Office 


Sulfur dioxide (S0 2 ), produced during the 
burning of fossil fuels and from volcanic 
eruptions, is a short lived gas which can act as 
pollutant near the surface with detrimental 
health and acidifying effects. With a mean life 
time of just a couple of days in the 
troposphere, emitted S0 2 is quickly converted 
to sulfate aerosol (S0 4 ) through oxidation by 
OH or by reaction with H 2 0 2 within clouds. The 
resulting S0 4 exerts a direct radiative effect on 
the atmosphere and it can also have an indirect 
radiative effect by inducing changes in cloud 
and precipitation microphysics. 


October 2005 
Sierra Negra 
Volcano 


MODIS 


GEOS-5 
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Column Concentrations of Sulfur Dioxide and Sulfate Aerosols 


The October 2005 eruption of the Sierra Negra Volcano on Isabella Island in the Galapagos Islands. A large plume of gases and steam was observed in this 
photo with NASA satellite imagery (top-left) on October 25, 2005. Image courtesy of MODIS, NASA. The plume of S0 2 and sulfate is seen being dispersed in 
the GEOS-5 7-km Nature Run (bottom-left) 



G MAO's NWP OSSE System "% 



Ron Errico, Nikki Prive. 
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AIRS Ch 295 QC-Accepted 




18UTC 12 July 






Observation Impact 



Radar W 
AMSU-B 
AMSU-A 
MSU 
AIRS 
HIRS 
RAOB W 
RAOB Q 
RAOB T 
ocean wind 
aircraft W 
aircraft T 
Satwind 
surf W 
surf Q 
surf T 
ocean Ps 
land Ps 



Adjoint Observation Impact 
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U-Wind RMS error: 
July 



U, 30N-90N 


U, 30S-90S 


U, 30S-30N 



RMS Error 


RMS Error 


RMS Error 


power 


Application: 
Analysis Diagnostic 




700 hPa T Analysis Error 



-180 -120 -60 0 60 120 180 

Longitude 
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MODIS Cloud & Aerosol 
Retrieval Simulator 



■ PDF based sub-grid 
sampling of GEOS-5 
fields (ICA) 

■ Spatial "clumping" 

■ Radiances for 27 
MODIS channels 

■ Cloud and aerosol 
extinction, ssa, phase 
function 

■ Operational Retrievals 

Clouds: MOD06 
Aerosols: MOD04 


a) Actual RGB composite b) Simulated RGB composite 



c) Actual SWIR composite d) Simulated SWIR composite 



Wind et al., 2013, GMD 
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Case Studies 

from WGNE Forecasting Exercise 





Tain I de faces 62099 

Tdtf fe qu-Krt: Jfii teff f-xes ttfij 
Numflro iraiiniD.de Tocos 1. 
Numefo iMximo de Tocos 307 
Numenj medio de focus 1U 
Desvio padrao dos focos 17.L 


62 099 hot spots 


i) Dust over Egypt: 4/2012 


2) Pollution in China: 1/2013 


3) Smoke in Brazil: 9/2012 
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Brazilian Smoke Case 

Simulated RGB Images 



Albedo Only 


Clouds and Atmosphere Only 


Everything 



Aerosol and Atmosphere Only 



Aqua Granule: 8 September 2012 17:30 UTC 




Brazilian Smoke Case 

Simulated MODIS Aerosol Retrievals 



Modis RGB Image with MYD04 retrieval overlay 

— i 1 1 1 1 1 1 r 
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Modis RGB Image with GEOS-5 aerosol overlay 
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Brazilian Smoke Case l&SA 

Simulated MODIS Aerosol Retrievals 


■ Retrievals tend to 
underestimated AOD 

■ Cloud contamination 
while present is not a 
major factor 

■ In the no-cloud 
simulation, surface 
inhomogeneity leads 
to data rejection 

■ Working with MODIS 
aerosol team on 
detailed diagnosis 


MCRS vs G EOS-5 Land AOD with clouds MCRS and GEOS- 5 AOD Land histoaram 



G EOS- 5 truth AOD Log( ADD + 0.01) 




Chinese Pollution Case 

Simulated RGB Images 




Albedo Only 


Clouds and Atmosphere Only 


Everything 



Aerosol and Atmosphere Only 


Aqua Granule: 13 January 2013 05:30 UTC 


M 




Chinese Pollution Case 

Simulated MODIS Aerosol Retrievals 



GEOS-5 AOD minus MYD04 AOD 


Modis RGB Image with MYD04 retrieval overlay 
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Modis RGB Image with GEOS-5 aerosol overlay 
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GEOS-5 aerosol species (WITH CLOUD) 
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Modis RGB Image with MYD04 and QA = 3 
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Modis RGB Image with GEOS-5 and MYD04 QA=3 



0 200 400 600 800 100012001400 


AOD value 


Chinese Pollution Case 

Simulated MODIS Aerosol Retrievals 



■ On average, relative 
small bias in retrieved 
AOD 

■ However, regional bias 
associated with species 

■ Cloud mask very 
effective in this case 





Saharan Dust Case 

Simulated RGB Images 



Albedo Only 




Clouds and Atmosphere Only 


Everything 


Aqua Granule: 17 April 2012 11:10 UTC 



Saharan Dust Case 

Simulated MODIS Aerosol Retrievals 



Modis RGB Image with MYD04 and QA=3 
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Modis RGB Image with GEOS-5 and MYD04 QA=3 
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Saharan Dust Case 

Simulated MODIS Aerosol Retrievals 



■ Low AOD bias over 
land 

■ Ocean results: too 
much or too little 

■ Cloud contamination is 
minimum forthis case 

■ Analysis in progress 




PACE Simulator 


■ GEOS-5 non- 
hydrostatic 7 km 
atmosphere with 
GOCART aerosols 

■ Coupled to GEOS-5 10 
km ocean component 
with biogeochemistry 

■ Simulation of 

Water leaving radiances 

■ (t.o.a. reflectances) 




Sinking 












Concluding Remarks 



■ A credible OSSE system requires well 
validated modeling components: 

■ Nature run 

■ Physical simulation of measurements 

■ Error modeling 

■ However, it must be validated as a 
System, by exercising it with the 
existing legacy observing system. 
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Nature Run Realism 



■ Global Domain 

■ Resolutions: 






GEOS-5 Global 
7 km Nature Run 



■ Components 

Atmospheric GCM on cubed- 
sphere, non-hydrostatic 

Prescribed SST, sea-ice 
Constituents 

■ Radiatively coupled aerosols 

■ Carbon species 

■ GMI Combo Chemistry (*) 

■ GEOSchem Chemistry 

■ Emissions 

Prescribed daily biomass-burning 
emissions (QFED) 

New dust source function from 
Ginoux 

Anthropogenic inventories 
downscaled to 10km 


■ GEOS-5 2014 NR 

Global, 7 km 

Aerosol, parameterized Chemistry 

■ ~ 2 yea rs simulation 

■ May 2005 - May 2007 

Aerosol, full chemistry 

■ ~ 1 month (TBD) 

Availability 

■ Free, on-line 

■ November 2014 

■ GEOS-5 2016+ NR 

Global, 3.5 km 
Improved model 
Cloud-aerosol microphysics, etc. 


(*) GMI combo chemistry used for short experiments only. 
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MODIS Retrieval Simulator 



■ GEOS-5 output sampled at MODIS Aqua geometry 

■ Surface albedo from MCD43GF Boston University 
operational dataset, available from MODAPS 

■ Ocean surface - MODIS Data Collection 6 Cox-Munk 

■ 24 spectral channels 

■ DISORT-5 simulation core 

■ Correlated-^transmittance model 

■ Rayleigh scattering included 

■ LiB radiance produced as if Aqua passed over 

■ IFF and MYD021KM files provided and IFF-compatible 
aerosol retrieval code executed over IFF files 

■ H-G phase function for aerosols, but g, w 0 and x from 
GEOS-5 



Aerosol OSSE Strategy 


Analysis Errors 


Adjoint Sensitivity 


Particle Shape 


Size Distribution 

Aerosol Models 

Speciation 


Index Refraction 



Surface 


Vertical Profile 



Nature Run 


GEOS- 5 


Cubed-sphere 


GOCART Aerosols 


10km Global 


Vector 

VLIDORT 



TOA Radiances 


s" 


VLIDORT 



Non-spherical dust 


Surface 


Land 


Ocean 


.* S.Obs Errors - 

X ' 

X. 

^ 


ML Estimates 


Cloud contamination 


Aerosol Retrievals 


Simulated 


Production Algorithm 
Synthetic Radiances 


AERONET Locations 


Cloud Screening 
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WGNE cases 



■ 5 cases from different areas dominated by specific aerosol 
type 

■ Sahara dust (1 granule) 

■ Biomass burning in Brazil (2 granules) 

■ Pollution in China (2 granules) 

■ Full GEOS-5 input files available for every pixel 

■ Contain atmospheric profile, cloud, aerosol information and 
other valuable data needed to start the simulation 

■ Radiance data available in following modes as IFF or 
MYD021KM files for all cases: 

■ Albedo only, no atmosphere, cloud or aerosol 

■ Clouds and atmosphere only, no aerosol 
Aerosol and atmosphere only, no clouds 

All constituents present: atmosphere, clouds and aerosol 



Level-1 Simulator 
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MERRAero UV Aerosol Index (July 2007) 
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Sampling Studies 
Level 2/3 Simulators 



Three snapshots of a significant aerosol event: 

Canadian forest fire smoke across the eastern US July 7, 2002 



Even after two months of averaging, the picture these different sensors provide of 
the regional aerosol load may be quite different. 



